Constructing E cient Decision Trees by Using Optimized Numeric Association Rules

We propose an extension of an entropy-based heuristic of Quinlan [Q93] for constructing a decision tree from a large database with many numeric attributes. Quinlan pointed out that his original method (as well as other existing methods) may be ine cient if any numeric attributes are strongly correlated. Our approach o ers one solution to this problem. For each pair of numeric attributes with strong correlation, we compute a two-dimensional association rule with respect to these attributes and the objective attribute of the decision tree. In particular, we consider a family R of grid-regions in the plane associated with the pair of attributes. For R 2 R, the data can be split into two classes: data inside R and data outside R. We compute the region R opt 2 R that minimizes the entropy of the splitting, and add the splitting associated with R opt (for each pair of strongly correlated attributes) to the set of candidate tests in Quinlan's entropy-based heuristic. We give e cient algorithms for cases in which R is (1) x-monotone connected regions, (2) basedmonotone regions, (3) rectangles, and (4) rectilinear convex regions. The algorithm for the rst case has been implemented as a subsystem of SONAR(System for Optimized Numeric Association Rules) developed by the authors. Tests show that our approach can create small-sized decision trees. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the VLDB copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Very Large Data Base Endowment. To copy otherwise, or to republish, requires a fee and/or special permission from the Endowment. Proceedings of the 22nd VLDB Conference Mumbai(Bombay), India, 1996